Papers with memory recall
Understanding Transformer Memorization Recall Through Idioms (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for analyzing memorization use definitions that are based on model performance, which changes between models and often also between training runs. |
| Approach: | They propose idioms as inputs that typically trigger memory recall and propose a set of English idiomas to test their methodological framework for probing and characterizing recall of memorized sequences in transformer LMs. |
| Outcome: | The proposed framework compares model behavior on memorized vs. non-memorized inputs across different model sizes and architectures. |
Disentangling Memory and Reasoning Ability in Large Language Models (2025.acl-long)
Copied to clipboard
Mingyu Jin, Weidi Luo, Sitao Cheng, Xinyi Wang, Wenyue Hua, Ruixiang Tang, William Yang Wang, Yongfeng Zhang
| Challenge: | Existing LLMs operate as an opaque process without explicit separation between knowledge retrieval and reasoning steps, making the decision-making process unclear and disorganized. |
| Approach: | They propose a language model inference paradigm that decomposes the complex inference process into two distinct and clear actions: (1) memory recall: which retrieves relevant knowledge, and (2) reasoning: which performs reasoning steps based on the recalled knowledge. |
| Outcome: | The proposed paradigm decomposes the inference process into two distinct and clear actions, memory and reason, guiding the model to distinguish between steps that require knowledge retrieval and those that involve reasoning. |
MADial-Bench: Towards Real-world Evaluation of Memory-Augmented Dialogue Generation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing evaluation metrics for memory-augmented dialogue systems lack practical value . current evaluation methods only consider passive memory retrieval while ignoring diverse memory recall with rich triggering factors. |
| Approach: | They propose to use long-term memory to create human-like dialogues using chatbots. |
| Outcome: | The proposed benchmark covers memory retrieval and memory recognition tasks with both passive and proactive memory recall data. |
TwinVoice: A Multi-dimensional Benchmark Towards Digital Twins via LLM Persona Simulation (2026.findings-acl)
Copied to clipboard
Bangde Du, Minghao Guo, Songming He, Ziyi Ye, Xi Zhu, Weihang Su, Shuqi Zhu, Yujia Zhou, Yongfeng Zhang, Qingyao Ai, Yiqun Liu
| Challenge: | Existing studies show that advanced LLMs produce text indistinguishable from human writing. |
| Approach: | They propose a benchmark to assess persona simulation across diverse contexts by decomposing the evaluation into six fundamental capabilities including opinion consistency, memory recall, logical reasoning, persona tone, and syntactic style. |
| Outcome: | The proposed model achieves moderate accuracy but falls short of the basic capabilities needed to simulate personas in real-world contexts. |
MMRC: A Large-Scale Benchmark for Understanding Multimodal Large Language Model in Real-World Conversation (2025.acl-long)
Copied to clipboard
Haochen Xue, Feilong Tang, Ming Hu, Yexin Liu, Qidong Huang, Yulong Li, Chengzhi Liu, Zhongxing Xu, Chong Zhang, Chun-Mei Feng, Yutong Xie, Imran Razzak, Zongyuan Ge, Jionglong Su, Junjun He, Yu Qiao
| Challenge: | Existing multimodal large language models lack the ability to memorize, recall, and reason in sustained interactions. |
| Approach: | They propose a multimodal real-world conversation benchmark for evaluating open-ended abilities of multimodal large language models. |
| Outcome: | The proposed benchmarks show that the models perform better in open-ended conversations. |
The Reasoning-Memorization Interplay in Language Models Is Mediated by a Single Direction (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models excel on a variety of reasoning benchmarks, but struggle to generalize to unseen questions due to over-reliance on memorized training examples. |
| Approach: | They propose to identify a set of linear features in the model’s residual stream that govern the balance between genuine reasoning and memory recall. |
| Outcome: | The proposed model can be manipulated to activate the most relevant problem-solving capabilities during answer generation. |